Decoupling of Modality Integration and Interaction Design for Multimodal Human-Robot Interfaces

The development of a multimodal Human-Robot Interface (HRI) involving mixed-initiative and context-awareness is complex and laborious. The integration of individual modalities (e.g., gesture recognition or speech output) and the design of natural human-robot interaction are two different tasks that both require their own expertise. In this paper, we consider three different roles that participate in the development of a multimodal User Interface (UI): the interaction designer, the modality integrator, and the multimodal UI designer. We present how the decoupling between these roles is facilitated by tools based on interaction modeling. We finally discuss how the decoupling is beneficial for introducing mixed-initiative and context-awareness in multimodal HRI.